1. Field of the Invention
The present invention relates to an image processing apparatus and, more particularly, to an image processing apparatus and an image processing method capable of performing good luminance correction without performing a cumbersome process. Also, the present invention relates to an image processing apparatus and an image processing method capable of performing good luminance correction without losing gradation of a high-luminance part.
2. Description of the Related Art
When a subject is captured under a biased illumination condition such as backlight, a large brightness difference may be generated according to exposure conditions of illumination and a bad image may be generated. Such an image is corrected by performing an image processing, thereby improving image quality.
As an image processing method, a Retinex process of extracting an illumination light component from an original image and correcting a luminance component of the original image by using the illumination light component has been known. The Retinex process is based on the Retinex theory wherein a human vision includes contrast constancy or color constancy, which is to see an external world at a state where illumination light is eliminated.
According to the Retinex theory, a human vision perceives a color according to a ratio of reflectance components of each object. The reflectance component is an image component of a subject, which does not depend on illumination. In contrast, in an original image captured by a video equipment, the value of each pixel is determined by the physical amount of received light and is expressed by a product of a reflectance component and an illumination light component. Accordingly, the reflectance component is obtained by separating the illumination light component from the original image, thereby obtaining an appropriate image which does not depend on the illumination light component.
As the Retinex process, various methods such as Single-Scale-Retinex (SSR), Multi-Scale-Retinex (MSR) or Linear Retinex (LR) are proposed. Here, for example, the LR method shown in Equation 1 will be briefly described. The LR method shown in Equation 1 performs correction of each component.
                                          R            i                    ⁡                      (                          x              ,              y                        )                          =                  A          *                      (                                                            I                  i                                ⁡                                  (                                      x                    ,                    y                                    )                                                                              Y                  ⁡                                      (                                          x                      ,                      y                                        )                                                  ⊗                                  F                  ⁡                                      (                                          x                      ,                      y                                        )                                                                        )                                              [                  Equation          ⁢                                          ⁢          1                ]            
In Equation 1, Ii(x, y) denotes a pixel value of a pixel (x, y) of an original image, Y(x, y) denotes a luminance component of the pixel (x, y) of the original image, and Ri(x, y) denotes a result of correcting the pixel value Ii(x,y) of the original image. i denotes each component of the original image I and becomes R, G and B if the original image I is represented by an RGB component. A denominator of a right side corresponds to an illumination light component and an image having a blurred luminance component is used as an estimated illumination light component. F denotes a filter function for planarizing the pixel (x, y) using peripheral pixels and may include a Gaussian filter or the like.
In Equation 1, since the luminance component of the pixel (x, y) of the original image is divided by the estimated illumination light component obtained by blurring the luminance component, the division result is distributed around 1. A of the right side is a gain correction value for associating this distribution with a luminance signal range, for example, 0 to 255. In the gain correction value, according to the need, an offset correction value is used or clipping is performed.
For example, it may be assumed the case where the luminance component of any original image has a distribution shown in FIG. 19(a). In this figure, a horizontal axis denotes a luminance value and a vertical axis denotes the number of pixels. Pixels are concentrated on a dark part with a low luminance value and a bright part with a high luminance value, thereby obtaining an image with a large luminance difference.
FIG. 19(b) shows a distribution of an estimated illumination light component obtained by blurring a luminance component of an original image and FIG. 19(c) shows a luminance distribution after dividing a luminance component of an original image by an estimated illumination light component.
In FIG. 19(c), values are concentrated on 1. The pixels are regarded as pixels in which the luminance component value of the original image is substantially equal to the estimated illumination light component value of the blurred image. In other words, the pixels are pixels of a low spatial frequency region with uniform brightness barely changed from those of peripheral pixels.
A pixel having a value close to 0 has low luminance and a peripheral pixel thereof has high luminance. A pixel having a large value has high luminance and a peripheral pixel thereof has low luminance. The peripheral pixel is corrected darkly in the former case and is corrected bright in the latter case.
As described above, in order to associate the luminance distribution after dividing the luminance component of the original image by the estimated illumination light component shown in FIG. 19(c) with the luminance signal range, for example 0 to 255, gain correction is performed. A lot of clipped highlights may occur or dark-part noise may be emphasized, according to gain settings, thereby influencing image quality. Thus, gain need to be appropriately set. However, since a luminance distribution shape or range differs according to images, it is difficult to set gain suitable for an image. Thus, processing is cumbersome.
In the low spatial frequency region with uniform brightness, since a value after dividing the luminance component by the estimated illumination light component becomes close to 1, regardless of the luminance of the original image, a bright region tends to become dark and a dark region tends to become bright. Accordingly, in a region in which a color change is low, such as sky, a phenomenon in which contrast of a low frequency region is deteriorated occurs.
Further, in a region in which a difference in luminance from a peripheral pixel is large, such as an edge part, since an estimated illumination light component is obtained by smoothing a luminance signal, a halo phenomenon in which a boundary part of a bright side is corrected to extremely high luminance and a boundary part of a dark side is corrected to extremely low luminance occurs. In particular, in a backlight image or the like, a halo phenomenon of a high luminance side in which a bright part of a boundary part between a main subject such as a person and a background becomes extremely bright is problematic. A method of suppressing a halo phenomenon by applying an edge preserving filter when calculating the estimated illumination light component has been proposed. However, in the edge preserving filter, the amount of calculation is large and thus processing becomes cumbersome.
Further, when an image is corrected bright, clipped highlights may occur in an originally bright part or gradation of a bright part may be lost. In order to prevent this phenomenon, for example, in Patent Document 3, a pixel in which an average luminance value of the pixel and peripheral pixels of the pixel is equal to or less than a threshold value is corrected bright by multiplying correction gain, and a high-luminance pixel in which an average luminance value of the pixel and peripheral pixels of the pixel is greater than the threshold value is not corrected by setting correction gain to 1 so as to maintain the gradation of the high-luminance part.
However, the detailed method of setting the threshold value is not disclosed in Patent Document 3. Patent Document 3 discloses that the threshold value is statistically or experimentally obtained and discloses only a special image clearly showing a luminance distribution. In practice, however, since the luminance distribution shape or range differs according to images, it is difficult to set a threshold suitable for an individual image.    [Patent Document 1] Japanese Patent Laid-Open Publication No. 2005-039458    [Patent Document 2] Japanese Patent Laid-Open Publication No. 2008-072450    [Patent Document 3] Japanese Patent Laid-Open Publication No. 2009-296210